The problem of writing error-free computer programs has plagued programmers since the beginning. Sophisticated schemes for automatically discovering program errors and bugs of all kinds, including lexical, syntactic, semantic and logical, have been developed.
Reliable and successful software is developed through sound, efficient and thorough testing. However, software testing is labor intensive and expensive and accounts for a substantial portion of commercial software development costs.
At the same time, software testing is critical and necessary to achieving quality software. Typically, software testing includes test case generation, test case execution and validation, and regression testing.
Most of software developers understand that the longer an error is allowed to remain in the software before it is discovered, the more expensive it becomes to fix. With vigilant testing from the earliest phases of the software lifecycle, software development teams find errors earlier and reduce the likelihood of discovering the error further down in the software lifecycle. Early detection reduces the cost of fixing each error, which in turn, reduces the cumulative cost of fixing the thousands of errors that typically surface in the course of a project.
For example, for testing of a complex web application, the application is deployed in a staging server environment for comprehensive testing before deployment to a live production environment. The web application is itself multi-tiered, complex and is comprised of other multi-tiered components. As such, integration and deployment of new versions of the web application on the staging environment is not trivial. In this context, the traditional feedback loop between quality assurance (QA) team and development team faces serious challenges.
In a software test cycle, when the software is first tested, a bug may be found and reported. The bug is then reproduced, fixed and verified to prevent it from occurring again. In an ideal setting, the QA team is able to verify that the requirements of the application are satisfied by the web application deployed on the staging server. Those requirements that are not met, for example bugs, are documented and the list of bugs is passed to development team. The development team then fixes these bugs. The web application with the bug fixes is re-deployed on the staging server and is verified by QA team to make sure that all the requirements are met. This can be an iterative process by which the number of bugs is reduced to zero.
What many companies face is a lengthy, drawn out cycle to re-deploy the web application on the staging server due to the complex nature of the web application and ensuing integration issues. As a result, it becomes difficult for the development team to correlate the bug with specific code to reproduce the bug, and to verify that the bug has been fixed. The development team is not able to test its code in the context of the web application. Even if this was possible, the number of times that the code is updated in the web application must be minimized. Therefore, there is a need for an effective way to reproduce, fix, and verify the bug outside the staging server environment.